Skip to content

[ExecuTorch][Vulkan] Add et_vk.q4gsw_requant kernel (STE re-quant to W_4X8)#20945

Open
JCNTH wants to merge 3 commits into
gh/JCNTH/84/basefrom
gh/JCNTH/84/head
Open

[ExecuTorch][Vulkan] Add et_vk.q4gsw_requant kernel (STE re-quant to W_4X8)#20945
JCNTH wants to merge 3 commits into
gh/JCNTH/84/basefrom
gh/JCNTH/84/head

Conversation

@JCNTH

@JCNTH JCNTH commented Jul 14, 2026

Copy link
Copy Markdown

Stack from ghstack (oldest at bottom):

Adds the Vulkan et_vk.q4gsw_requant kernel — straight-through re-quant of fp32 latent weights into the frozen-scale 4-bit codes, for on-device weight training. Writes the codes directly in the forward's W_4X8 layout so no per-step re-pack is needed.

Problem: et_vk.q4gsw_requant is registered in the shared Vulkan partitioner but Vulkan had no runtime kernel.

Solution: a GLSL kernel that quantizes round(latent / scale) clamped to [-8, 7] and packs the codes in the W_4X8 block layout the forward reads, reusing the exact byte-pair convention of glsl/pack_q4_linear_weight__w_4x8.glsl (even-N low nibble, odd-N high; one ivec4 per 4K x 8N block at (k4, n8); OOB upper tile = the bias-zero 0x88888888). A zero scale yields code 8 (no divide-by-zero), matching the eager reference.

Key changes:

  • glsl/q4gsw_requant.{glsl,yaml} — buffer x float; 2D dispatch over (k4, n8).
  • impl/QuantizedLinearRequant.cpp — prepacks the constant scales; output is the W_4X8 int buffer [K4 * N4_padded * 2]; group_size spec constant; dispatch-grid guard.

Design note (for review): per the layout decision, requant writes W_4X8 to match the Vulkan forward. Today the forward and backward each prepack their own flat [N, K/2] weight internally, so nothing yet consumes an externally-produced W_4X8 buffer — closing the training loop needs a forward path that reads a mutable pre-packed weight (the weight-lifecycle follow-up). The op's AOT meta in custom_ops_lib.py still describes the flat [N, K/2] output and should be reconciled with this W_4X8 output.

Constraints: buffer storage, fp32 latent/scales; N % 4 == 0, K % 4 == 0, group_size % 4 == 0.
@exported-using-ghexport

Differential Revision: D111797527

Differential Revision: D111797527

[ghstack-poisoned]
@JCNTH JCNTH requested a review from SS-JIA as a code owner July 14, 2026 21:50
@pytorch-bot pytorch-bot Bot added the module: vulkan Issues related to the Vulkan delegate and code under backends/vulkan/ label Jul 14, 2026
@pytorch-bot

pytorch-bot Bot commented Jul 14, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20945

Note: Links to docs will display an error until the docs builds have been completed.

❌ 10 New Failures

As of commit 2895d46 with merge base 7013c8d (image):

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This was referenced Jul 14, 2026
@github-actions

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 14, 2026
[ghstack-poisoned]
[ghstack-poisoned]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. meta-exported module: vulkan Issues related to the Vulkan delegate and code under backends/vulkan/

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant